POMDP solution methods
نویسنده
چکیده
This is an overview of partially observable Markov decision processes (POMDPs). We describe POMDP value and policy iteration as well as gradient ascent algorithms. The emphasis is on solution methods that work directly in the space of policies.
منابع مشابه
COMP-627 Project Belief State Space Compression for Bayes-Adaptive POMDPs
Partially Observable Markov Decision Processes (POMDP) provide a nice mathematical framework for sequential decision making in partially observable stochastic environments. While it is generally assumed that the POMDP model is known, this is rarely the case in practice, as the parameters of the model must be finely tuned to reflect the reality as close as possible. Hence it is of crucial import...
متن کاملAn Online POMDP Solver for Uncertainty Planning in Dynamic Environment
Motion planning under uncertainty is important for reliable robot operations in uncertain and dynamic environments. Partially Observable Markov Decision Process (POMDP) is a general and systematic framework for motion planning under uncertainty. To cope with dynamic environment well, we often need to modify the POMDP model during runtime. However, despite recent tremendous advances in POMDP pla...
متن کاملPoint-Based Policy Transformation: Adapting Policy to Changing POMDP Models
Motion planning under uncertainty that can efficiently take into account changes in the environment is critical for robots to operate reliably in our living spaces. Partially Observable Markov Decision Process (POMDP) provides a systematic and general framework for motion planning under uncertainty. Point-based POMDP has advanced POMDP planning tremendously over the past few years, enabling POM...
متن کاملFinding Optimal POMDP Controllers Using Quadratically Constrained Linear Programs
Developing scalable algorithms for solving partially observable Markov decision processes (POMDPs) is an important challenge. One promising approach is based on representing POMDP policies as finite-state controllers. This method has been used successfully to address the intractable memory requirements of POMDP algorithms. We illustrate some fundamental theoretical limitations of existing techn...
متن کاملPOMDP Planning by Marginal-MAP Probabilistic Inference in Generative Models
While most current POMDP planning methods have focused on the development of scalable approximate algorithms, they often neglect the important aspect of solution quality and sacrifice performance guarantees to improve efficiency. In contrast, we propose a novel approach to optimize POMDP controllers by probabilistic inference while obtaining bounds on solution quality as follows: 1) re-formulat...
متن کامل